BO IBB 5$1 

TITLE 

TNSTTTOTfOtj' 

POB DATE 
NOTE 

'eCFS Pl^ICE 
DESCFIPTO^^S 



« 



DOCOREHT RESUHE 



922 



BrowTi, <,S*.ep!ier: Nnthensor., Michael ^ * 
DesianiTiq "Instructional Mi-^re-^iijls: Guesswork or. 
Facts?' .- 

Open- OT\iv. . waltoTs-, Bl'etchlev» Bucks (England). * I'^p' 
of Educational Techno'lcay, 
Nov 78 



IDENTIFIERS V 



SPSTPACT 



HF01/PC02 Plus Postaae. 

♦ Academic Al^ility: *Pasicf Skills: *External ' Pearee 
Proorams: ForeiaT>* Countries; Riqh^r 'Educatior. : 

♦ Instruciiop.al Ma^^rials: lns,tructional Systems: 
Ti^troductorv Courses: larauaae Skills: Mathematics 
Anxiety: ♦Science Proora n>s: - *Social Sciences: 
Techniiajl Education < . - 

♦Open University JGri^a^ Britain) - 



A* systematic study is reported , of the mathem'atical 
skills ?ind learnina abilities of new students immediately prior to 
entry into the'open Oniversi^y's (EnqlaT^d)' foundation courses in 
technoloqv and social sciences, ^^he iT^^en^ was to provide predictive 
informatic.n about potential student^ entry skills fo^r the team 
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Bii^ematical skills were assessed by a que*tior.ra/r<fe-test given to'=> 
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♦tte skills in the course. 1^1-^ concluded tha+ such- systematic 
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the Open Oni vers.it y and in other institutions. (MSE) 



\7 



/ 



♦♦♦♦♦♦♦♦♦^♦♦♦♦♦♦★♦^♦★♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦^^♦♦♦♦♦♦♦♦♦♦♦** ****** ****** 

*1 it peprcductions supplied by EDPS are. the best that can be made * 
' « ' ^rOffl the oriq1.nal document, * 

♦*♦«*<♦♦♦♦«*♦♦♦♦♦*♦**♦***♦♦♦♦♦♦♦♦♦♦♦♦♦**♦********♦♦**************.****** 



o 

ERIC 



ABSTRACT 



o 

This is a report^ of a systematic study of the 
matheaiatical skills and learning capabilitiea of new 
students icmediately prior to entry into.TliiOO and 
DlOl^ the Open University. Foundation Courses in 
Technology and Social Sciences resp^tively* The 



aim of the Study was*^to prcuvide predictive 

information fibout potential 9tudent entry skills for 

* 

the Course Team preparing TtOl^ the n^ Technology 
Foundation Course* 



The report describes %he raethtod used in the study 

» 

and explains how i^t fonoa part of an overall ^trategy 
^ for ensuring that the design of the instructional 
material to be used in TtOl is based onithe known. • 
IcMkrnXng capai>ilities and requirements o^ its likely 
students.* Th^findings of the study are reported^ ^ 
briefly and the pr6bl^ms and advantage of using this 
kind of a|iproach to cuVnriculum development are 
discussed^ 
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BACKGROUND TO THE STJUDY ^ 

Undergraduates of the Open. University are normally required 
to take two foundation courses towards their degree. Bach 
foundation course is considered to be^the equivalent of||psix 
^ months fult^-time study or one year part-time* There are five 
foundation courses^ one for each of the following faculties: 
ArVs,. Social Sciences, Science, Mathematics, Technology* 

' A major task for foundation courses' is to bridge the gap^ 
between school 'and university lel^el study in terras of student^ 
study, communication and dialectical ski^s. The Open University 
was crated to providp educational opportunities for people who 
have missed out for some reason on the ediicati bo facilities 
avaijlable to ^hem in earlier life* Consequently, Open* University 
students come from a wide range of backgrounds, employment and 



education, and th^ can differ widely on ^ try in terms of age^ 

' V 

skills and knowledge. A distinctive feature bf foundation courses 

therefore must be their breadth 6f appeal and their ability to 

meet t;he educationaf. req^ir^ents of an extr«nely diverse student 

. population* " 

■ ■ / V, 
I TtOl is the code number of the new Open University Technology 

Foundation Course entitled "Living with ^Technology It is planrtfed 

.t 

to replace the existing course (TlOO': The Man-Made World ) in 198t^* 
TiOl is a course about technology irather than in technolSgy in the 
sense that it is conoemed with the ways in which technology 
influences both our present way of life and. our future. The team 
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redpoftiible for producipo the course, the Course. Team, belxe^res 
.K..S.i^in ff with Technology will be of vitai interest and relevance 

^ broad spectrum of students, not ^lut those with an interest 
in the engineering aspects of technology. Consequently, the Course 
Team vants to present the coui-s^ in a way which will make it 
accessible to studints yhaWer their previous background or skills. 

A basic and, on the' face of it self-evident, tenet of good^^ 
teachino is that the t^ching material should be matched to .the ^ 
skills and learning abilities of the students at whom it is * . 
directed. That is to say, the teaching material should begin at a 
level which correspond, to the skills of the students immediately 
prior to taking the course and it should proceed at ) pace 
consonant wil6i their, abilities to assimilate W new material. ^. 

^ * • 

Deiipite the obvious coBWon sense of this approach it is not 
•always carried out in practive for a v6ry good reason. NAt tl^e 
time the teaching material is being prepared neither the entnr 
Skills nor the-4.ean>ing cabililities of the future students 



f 

known, i 

I 



In the case of Tt(W, the. target student population for the new 
Technology Foundation.^m;»e is broader than that currently 
registered for TlOO Jfrlble ' 1 shows that only 11% of the 
applicationp from L student* for fom&Atlon courses 'are for ,T100. 
it this Course Tea. is successful a much larger proportion of 
studsnts who currently apply for other foundation courses in the 
University (i.e., in the Soiial Science, Science, Arts and 
Mathematics Faculties) will also apply for^TlOl. \^ ^ 



Applications from nevSr students ftor Foundation Courses 
averaged ovef^' 1977-*! 979. (Note the percentage' figures 
quoted are approximate only J^o the nearest integer.) 



.Foundation 
Course 


^ , Applications ^ 
% 


Arta 

• 


. 34 
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Social 
Sci^Kxe ^ 


26 • 


1 

1 

Science 

-—^ 




Maths 

t f 


— " — :r- 

14 


Technology 





'Detailed infonsation on student €Ut>ilitie8 prior to course 
conaencenent is not availat>le« The only available ^institutional 
, reseSrch data list student characteristic^ such an age^ occupation, 

* • ^ 

seXf ^i^evious qualifications^ atc^* or describe student, performance 
on a particular course e.g., nuinber of assignments passed, number 
of. students taking the exam, average exam scores, etc* Ifiiat these 
kinds of data do not tell the course designer 4is. wh^t skills tl>e, > 
students are likely to possess at tAe 1t>eginning pf*the course, or 
how they ^wi 11 respond to diff^ent teaching strategies. In other 
words, the existing suyv^ data are? not sufficiently pi^edictive . 

To overcome these difficulties the Course Team has decided to 
adopt a threerI^?u^t _ strategy aimed at ensuring ^that the course 
material is tailored as closely as possible to the learning 
reqxiireoents of its studmt^* • 

t 

The ^irst part of the strategy is a skills survey aimed at 
providing quantitative^information about the likely entry skills 
and learning abilitil^|pf T101 studwts. The intended function of 
this information is to provide some guideline^ for setting 
'standards in'^an area fraught with Uncertainty, conflicting opinions 
and myths. > ^ * * « 

Thil second part is a scheme for developmwtally tSstiiig the 

r * 

teachings material for the first six stiidy weeks of %he course in 
draft form. The first six study weeks are curcual to successful 



study of the entire ctiurse since they vrLll lay the foundations ^ 

t . ■ ^ 
of 'the "basic numeracy and literacy skills in addition to 

introducing students to some importan'^ technological activity* 

areas. "The results obtained from the developmental testing will 

be used to ratify or revise as necessary the findings from the 

skills survey. J . • 
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The third part of th$ str^^^gy is. d^ong-term course evaluation 
prograoine. with the aim of assessing the effectiveness of the 
teaching in the coursie*and retrospjectively identifying learning 

dif fi;culties encountered by the students. It is recognised that 

' ■ *■ f 

the previous two* approaches (a pre-course skills survey and . 

r • 

devQ^pmefital testing of parts of the course) although' useful ^f or 
e;6tabli sluing reference points,-* are unliftely in themselves' to 
enable the Course Team to produde'the best possible cours^e on a 
first attempt^/- It will be ^^necessary to assess^ the effectiveness 
of^j|^e course as a whole in its Yinal polished form apd toXmake 
modifications^ tc it For this reason a Course Evaluation Team has 
been estabHsHtH with the responaibiUty for collecting feedback 

irj^j/yeai^oi 

presentation. ,(4980). The resulting feedback data, will be the basis 
for modifying the course as necessary during the second year (1981), 
and for re-presenting a revised course in the. third year (1982). 
It is hoped that €he reyised course will be matched sufficiently 
clos^y to student)^ • learning requirements for it to be able to 
run substantiaHy unc^hangeKl fol- six years. . 

In due cou]:se reports will be , produced discu^sing^^J\e- results 
of *the second emd third parts of the Course Team strategy. Thip 



from students on all ai^ects of the coursfe during its firj^^year^f 



papfr is concerned with tho fdrst part: the lis Survey. 

THE SURVEY METHOD * ^ 

' # ■ 

l?he aim of the gurvey 

The two skills of literacy and numeracy referred to* above were 
identified as being of importance to the? study of technology • Of 
these t ntimeracy was felt to be a particularly important topic for 
investiga^im^ because the ability to solve problems numerically is 
considered to be one of the most important skills of technologists; 
it is also a skill which has caused difficulties for students on 
tlie curx^t Technology Foundation Course. Moreova*, within the 
Open University as a whole« mathematics se«n to be a stumbling 



ock for students in as much as courses with substantial mathematlQS 



have hig*her withdrawal rates than courses with littl^ or nonef More 

fundamental ev^ than basic numeracy are the algorithmic skills 

V^equired to carry out the computational involved once the problems 

have been appropriately (Expressed. The aim of the survey was to 

provide the Course Team with det^irled useful iriformati^n about 

» 

students* entry level algorithmic 6kill£i and learning capabilities* 
in areas relevant to 'the suf>ject matter . proposed for the beginning 
of the courage: ^ 

(t) Basic Arithmetic, including addition, subtraction, 

multiplication and divteion* negative numbers, precedence" 

ft 

of operators 'and brackets. . 

g » .... 

(2) Fractions and Ratios, including addition/ subtraction? 

multiplication vand diviaiion of firacti^ns and, the cpnvei:sion 
*■ '* * 

of fractions 'to ratios And vioe<^versa. ^ 



(l) Percentages, including calculatin| percentages, 

convert*! ng percentages to fractions and vice-versa and 
percentages to decimals and vice-versa. 

(4) Decitnala, including converting fractions to decimals 

and vice-'versa, multiplication and division of decimals 

«> ... 

and rounding, truncating and^ significant figtires; 

(5) Areas aiKl Volumes, including calculating the areas of 
rectangles, parallelograms, triangles and circles, the 
surface areas of simple solids and the« volumes of simple 
solid9« 

• . . . ... — 

(6) Graphs, including plotting and reading coordinates and 

« 

plotting graphs • 



Vithitlitech ^f these six areas the Course Team wanted answers 



esfrohs < 



to four ques 

(1) *Vhat proportion of the sample target population are able to 
carry out simple calculations without €uiy help from TtOi? 

(2) Vhat proportion of the sample are able to perform the 

calculations with oftly a •brief reminder* 7 

« 

(l) Vhat proportion of the sample required detailed teac};]iing in 
order to learn (or releaWi) the algorithmic skills reqviired 
f to perform the calculations? 

♦ 

ik} ' How much time im apent studying the detailed teaching? ^ 
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A 3dparatQ questionnaire was prepared for each of the six 

areas and was divided into a nuinb^ of blocks of calculations. 

Each block covered three or more subskills of the general skill 

ft 

being assessed • 

* ■ * 

For example, in the Bastt: Arithmetic questionnaire, the first' 
block assessed simple addition, subtraction, multiplicationf and 
division; the second assessed the use of * negative numbers*; the 
third * precedence of operators and the fourth block assessed the ^ 
use of brackets* 

Students were asked to tackle all the bloclcs of calculations at 
the beginning of a questionnaire, then, to mark theii^ answers to the 
first block. If they got them all right they proceeded to mark 
their amswers' to the second block. If they- got any wrong or were 
unable to do any of the calculations in the first block they were 
directed to a brief reminder of how to do those- kinds of calculations 
then to another similar set of calculations to tacfcle. If they got^ 
all the calculations right a]^ter the brief r^iinder they were asked^ 
to proceed with* marking their answers to the second block. Otherwise 
they were asked to study some detailed teaching and to attempt a 
further set oiT* calculations. The detailed teaching, material used 
was partly culled from a variety of existing OU and other sources ' 
and partly prepared espjBctHlly for the survey where none of the 
existing material seemed suitable. After marking .their answers 
to the 'third set of calculations t,hey were asked tp proceed with 
narking their answers to the next block. '?his procad\ire, which was 

♦ 

re])eated for each block in turn, is sutomarised in Figure 1. 
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Fig. 1. Procedure for completing questionnaire 
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students were also asked to record the amount of time they 

sp^nt studyitig the detailed teaching and answering the problems 

*' - • - 

which followed it. 7 - 

Selectiott^df the sampl»^ population * - T 

There wasi at the ti%ie of carrying out the survey, no pool of 
applicants for the course v^ich could be sampled fpr their pre-cburse 
skills or learning capabilities. Such a pool will not^ be available 
until late 1979t i.e., when the course will be almost ready for 
presentaticlh* Instead a sample population h^d to be defined which 
would be as representative as possible of th6 kinds of student tlOl 
is intended to attract* The target student population w€is 

identified as the present range of students who apply for T10O plus 

• * ■ 

a larger proportion than at present of studrafts who are attracted 

to the Social Sciera:e Foundation Course,* DlOl. This is not to say 

that the Course Team hDpe'tq lure students away from the Social 

Sciences but rather it is hoped that more students t^ill take both a 

Social Scieribe and a Technology Foundation Course. It was decided 

therefore to look at studraits who had been offered a place on either 

TldO 6r D101 and who were waiting for their course to beg^n. Only new 

students (i.e#$ those commencing their firi^t year. of s^udy as Open 

University undergraduates) were regarded as relevant subjects for 

■ . •• , . 

the study. ^ This was because TlOl will be oriented primarily .towards 

. ■ •* . 

the requirements of students with no previous experience of university 
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Table 2. Percentage Withdrawal of New Students from TlOO at 
3'1. t2". 77 by Educational Qualifications. 



Educational 

cations • ' «^ 


Percentage of students 
in this* cateaorv who 
have withdrawn from 
the course 

—C — «. 


No forma]. 


45 


cse/rsa 


50 


•0« level 1-4 


50 


» 0» level 5+ 


28 ^. 


»A» level 1 


22 


•A» 2+ 


f 

22 ■ 


* 

CMC pr QND 


24 


HNC or HMD 


16 


Teachers Cert. 


13 


Univ. Diploma 


15 


Univ. Degree 


17 

•* 


No data 
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l^el study. Consequently, it is necessary to pase the design of 

r . 

the teaching Inaterials on the skills £^nd capabilities of new 
students rather than those already in the O^pn University Teaming 
system* - • ' ' ' 

• * • * 

It w^? decided not to survey the vhole body of new students 
^ . ..." t ■ 

registered for both TlOO and D101> since many of those can be 
expected to possess quite cTophisticated numeracy skills. The aim * 

of the study was to identify 'the lo\«est likely levels of entry 

' ' ^ ' f ' ' 

skills and learning capabilities of potential TlOl students in order 

to determine the starting level' and learning gradient for such skills 



in the cour&re. Applicants for Open University courses are asked to 
indicate which of a number of educational qualification categories^ 
they fit into pn entry. Tabl'e 2 lists these educational categories 
in the left-hand column • 



^ Thfe Table shows the^ proportj^on of new stude«its who had iirtthdrawn 
from TICK) in 1977 by the end of the course' expressed as *a percentage, 
of the total number of new students who had paid their course fees. 
From this Tabl^ it can be seen that there is a tendency for students 

with few educational qualifications to withdraw;- from TlOO in greater 

'# •■ * • 

proportions -l^han those with rather more qualifications. 

It waaldecided to include students in the lowest three categories 
only in the^ample. Th^ sample population therefore was all 2279 
of the new students who had registered for either TlOO or DlOl in 1978 



atwl vfho fell into one of the fo^llo*ringf .three educational categories: 

-^^ • . ' ■ 

(l) Students with no formal educational qualifications. 

(2") Students having a CSE, RSA or school leaving • 

ce^^ific^te in one or more subjects. • ' 

(l) students with QCE '0> level, CSE (Q* grade, school 

♦ 

certificate or equivalent in 1-4 subjects* 

Table 3 shows the proportioiis of students in each of these 
categories as a percentage of the total intake of newly registered 
students to bo,th TlOO and DlOl in 1977 • It is apparent from the 
vTable that thp proportions were approximately the same for both cases 



Table >. Proportions of Newly Registered Student Intake in 1977 

V.HI. * 

in Different Educational Categories. 



Educational 
Qualifications 


COURSES 


TlOO 


DlOl 


No formal 


11* 


10% 








1-4 '0^ levels 


13% 


.10% 


" Total i 


r 


24% 



Administration of the survey 

It was decided that it would' be impracticable to ask any student 

* *" " ■ , 

to attempt more than onb questionnaire because of the amount of timo 

^ * t 

requlr^ to answer each one. Pilot tests indicated that, for those 

students who fiad to work through all* the detailed teaching and the 
accompanying problems , each questionnaire required approximately^ 
24 hour? study time to complete. Accordingly i/the sample population 
was split into six equally siaAad groups, each group receiving a 
different queaiionnaire. Within each group tha thi:ee educational 
categories sampled were represented in the same proportions as in . 
the sample popula,tion as et whole. Each qi^stionnaire was accompanied 
by a covering letter which explained the aims of the survey. It also 
invited the recepient to return the questionnaire blank if ,they were 
unable to complete it for som^ reason, "but to specify what those 
reasons were.* No inducement, t^ respond was offered oth«r than 
an appeal to their* altruii^m\ ^ ' 



The first J^wo. questionnaires were mailed^ to students ih NoVewiber 
1977 and the remainder were sent As as they were ready. The 

last one was despatched at^the beginning of February 1978. ^ 

•* c 

. THE SURVEV^RESUM^S j' * 

Response rate * ^ 

I' 

The response r^te for oompleted returned questionnaires declineii 
gradually from 66^^ to 5796 with a mean response rate, of approapimately 
62}t. This decline can probably be explained ^y the fact th4t 
foundation courses f^egin in early FAruary and in additioii to the 



actual eourfte textft, studmts receive a large volume of material ^ 
from their regional office and other OU sources. Conse|iu«nt*ly, the 



later questionnaires would have been in increasjing competition with 
other more pressing material for the litudents* attention. 



Of the 389^ who did not return a completed questionnaire, somr 
responded to our invitation to retiurn it blank with an explanation* 
These responses were bolstered by ^additional comments volynteered 
by students who had returned completed questionnaires. The inter- 
pretation and classification of such open-CTided ^feedback was to a 
large extent arbitrary* Many responses contained^ multiple reasons 
for non-return. However the following four categories .emerged as • 
the major relevant ones:- 

(1) those who regarded the exercises as too daunting to attempt J 
and those vho attempted them anyway and wrote to us telling 

us how difficult they were (28 responses). 

r 

(2) Those wt^o for personal reasons such as illness^ changing job, 

f 

^^((pioving home, etc. were unable to find the time to complete- 



the quesi&i^nnaire (2| responses). 



* 



(3) Those v^o did not understand what they had been cusked to 

^ ■ — 

do'(9 respons^). #^ 

(4) Those who regarded *he exercises *set as too trivial to be 

■ s - . * 

worthy of an attempt and those who attempted them anyway 

but *rrote to us telling us how easy they were (7 responses) 



The misunderstanding^ referred to in category 3 above were 
almost' entirely due to the fact 'that the DtO:|i^stud«its who received 

a questionnaire assumed that we had sent it to them by mistake. Many 

t- - • 

1 ' . • 

\: of these respondents told us that their difficulties with 

• ' ' ' » ' , ■ 

mathematics had^ contributed* to their decision never to take a 
technoJLogy course. This is inter i^ting because such students are 
precisely the kind we are tr;j|fing to encourage to study TtOl. 
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Fig.. 2. Performance differences Associated with Course Preference 
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Performance <Ufferences associated with courge preference 

"7 ^ ' ^^^^^ 7^ 

Djlfferences between the TlCX) and D^Ol students are st^vm i« 
Figure 2. It can be seen f roim the figure that on average about 15% 
ii»re TtOO studmts than D104 students got all the |gjroblems right on 
their first try. In most cases this differ Mice was reduced to about 
• 9% by the reminder and further reduced to about 6% by the detailed 

teaching. The two exceptions to this trend were the Fractions and 

Ratios questionnaire and the Percentages questionnaire. Oti the 

Fractions and Ratios quest iomiaire, the initial difference b^ween 

TlOO and DlOl students waaTTairly small (8^6) and it^ ihowed little 

change after either the ^^ninder of the detailed teaching. On the 

T 

Percentages questionnaire* a larger initial differerfce" of was 



reduced to l6% by the reminder* and to 11% by the detailed teaching. 



Performance differen ces associated vith educational qualifications 
' — « '. . 1 

•■ / ^ ' , . ■ 

* Performance differmces associated vrith differences in educational 
qualifications are shown in Figure 3« On four of the sfx questionnaires, 
a larger proportipn of respondents in the l--^ 'O* level category got 
all i»he answers right on the^^r first tJtV than did respondents in 
either of the other two categories| on the Graf>h& questiotii^ire, the 
differmoQ between the 1-4 'levels^ and the ••no qualifications'' 
category was only 1%-* On the two questionnaires where the performance 
of the 1-4 •0» levels grdup was not the best. Decimals and Areas ^nd 
Vol tones, the differences, between all three categories were leas than 

!%• Surprisinglyy respondents iflth no formal qualifications did 

*« ■ ■ 

slightly better than students in the CSE/RSA category pn the ^i rat 

t 4 

.try in five pf the six questionnaires • » ' . 

There was some tendency for* pa*fomemce differences between • 



ERIC 



22 



% of students getting all the problems right 



100- 



90- 



80- 



70- 



60- . 



50- 



40- 



30- 



10- 



TRY 1' 2 3 1 
> * No formal 



'2 



BASIC ARITHMETIC 



t 2 .3 TRY. 
1-4 levels 



Fig. 3^. Perfonaancfe Differences Associated with Educational 
Qualifications. 



23 
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categories to be reduced on successive tries , moi^t notably in' the 

» 

Basic Arithmetic queaitionnaire. It is important to note, however, 
that in the Percentages quest ionnai re , the initial 109^ lead 
established by the 1t4 'O* levels jjroup was barely diminished 
despite the fact that the other two groups converged* Similarly 
in the Graphs questionnaire, the 1--4 *0* levels and the no 
qualifications groups began with^ an approximately S% lead over the 
CSE/RSA group, a lead idiich thecT retained throughout successive tries^ 

Overall perforwance results 

Figure 4 summarises the overall performance results of the dtudy. 
For each questionnaire, it shows the proportion of respondents who 
got all of. the problems right on the first try, after the reminder, 
and following the detailed teaching. 

\ 

It can be seen from Figure 4 that the remir^ders* were in all cases 
effective, producing on average, a 20j6 increase in the proportion 
of respondents ^le to get all the problems right on their seopnd 
try. The effects of the detailed teaching were less clear* Although 
moderately effective on the Basic Arithmetic and Decimals questionnaire 
the overall increase in the proportion of students able to get all the 
problems right after the detailed 'teaching, was on average, less than 
5%; This left some lOfS of the respondents still imable to answer the 
problems correctly, even after being exposed to tl^ie detailed teaching* 

Overall approximately 3?% of tfie respondmts did not get alX the 
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problama right on their first try. Assuming the respondents to he 
representative of the sample population, this suggests that 15% 
of all new T100 aria DtOI students in the three educational 

categories sanpled need some help with the kinds of problems set. 

i' • ■' ' 

From Table 3 itflsan be seen that students in the educational 
categories^ sampled constitute 26% of the oo^ined intake of new 
studtets ^o TXOO and^DlOl. Thus the proportion of new students 
entering these courses who need helprWith basic mathematical skills _ 
is 35% of 26% or just over 9% of the total intake of new students 
into TlOO and DlOl. 

DISCySSIGN OF THE RESULTS 

In general the results of the study indicate that the algorithmic 

skills present difficulties for a significant proportion of students 

in the categories sampled, particularly for those with few or no 

academic qualifications and those who have indicated a preference 

for social science rather than technology courses. The fact that 

some (38%) of the students selected for the survey sample did not 

^respond obviously has implications for the representativeness of the 

results obtained and for the validity of any conclusions drawn from 

those results. Although meagre^, the open-ended feedback suggests 

' that a major reason for non-response to the survey was a desire by 

individuals not to reveal their ladk of numeracy skills. The 

feedback also suggests that students with numeracy problems 

deliberately avoid the present Technology Foundation Course precisely 
♦ 

because of this weakness. In other words, the performance of those 
students who responded is probably better on average than the actual 
mean performvice capability of the whole sample. In addition, it 
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seems 'reasonable to suppose that numeracy problems are not confined 
to students' in the categories sampled and that the problem is 
even greater thkn indicated by the survey results* For example ^ 
students with^-only one^*A« level or ev€(n a degree in an area not 
requiring numeracy may have probl^ns wit^h math^aatics. Similarly, 
students who currently register for the Arts Foundation Course may 
do so .partly because they are reassured by its essentially non^- 
mathematical Syllabus. 

The ineffectiveness of the detailed teaching used in^||||pte survey 
is of particular concern to curri<^luiai designers. As stated in the 
introduction^ the matldrial was dravm partly from a range "of Open 
University sources, and partly generated especially for the survey. 
'One obvious problem with the detailed teaching was the absence of 
practise exercises* Several respondents remarked that they believed 
they could have* done better given more time to practice the skills 
being taught. Others actually told us that they had borrowed "Teach 
Yourself Mathematics*' types of books and set themselves practise 
exercises.- 

Applying a 90% criterion-level to the results it has been possible 
to make firm reconnmdations to the Course Team ab6ut the level of < 
teaching which should be provided in TlOl for each of the skills ^ 
surveyed. These recommendations^ correspond to the throe levels of 
teaching provided in the questionnaires: none; a brief r^ainder^ ^ 
detailed teaching. Thus if or more of the respondents got all . 
of the problems in a particular block right on the first try then 

i 

the' recommendation for {providing no teaching was made. If less than 
90jt got all the problems in a block right on the first try then the 
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recoinmendati<Ai was for the provision of a brief reminder* Similarly 
if less than 90J6 of thearespondents who tried the problans after the 
reminder got th^^ll right then a recoBnnendation was made to the 
Course Team for detailed teaching of the skill covered in that block. 

In the light of the survey finding that on average 10?6 of the 
respondents were still urtSble to get all the problems Tight after 
the detailed t^aching^ four further recommendations have been made:- 

V 

1 

m 

(1) Numeracy teaching should be integrated into the. course material 
in a meaningful way. Because the course is designed to be 
issue and problem oriented, the need for calculations will 
arise naturally from topics discussed, concepts taught, and 
evidence evaluated. It is anticipated that this integration 
will be more motivating, than for example, the kind of 
preparatory mathematics booklets used on other Foundation Courses. 

. . 4 ^ ' 

(2) Mathematical topics should be taught via the use of a calculator. 
This will allow students to tackle problans using realistic 
data right from the start without having to go through the 
tedious process of long computations. It ^s self-evident that 
students will have to be taught how to use the calculator in 

a sensible manner. 

(3) The first six weeks worth of course work which ccintaina the 
bulk of the basic mmieracy teaching should be mailed to students 
early, so that instead of mix we^cs new students will have the 
option of spending two to three months practising and^-conaolidating 
their basic numeracy skills. _^ 

(4) Optional practise exercises should be provided for the minority 

■ '■' ^ ♦ 3.1 
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of students who still have difficulty after the detailed 

teaehing but it should be stressed that'"^ these are additional 

to the course workload. That is to say, the Course Team shoul ' 

not take the learning requir^ients of these students as its 

baseline for developing numeracy skills but it should provide 

extra optional help for the few who need it* 

i 

A number of important assumptions underlie the wa^ in whiqh the 
Numeracy Study has been carried out. These concern the representativeness 
of the sample selected, the extent to which t^e material used in the 
questionnaires is likely to correspond to the teaching material used 
in the course, the validity of the comparisons drawn between different 
groups within sample and the reliability of the data obtained • 

It has been assumed that the selected Sample is representative 

o^ the future student intab:e into the new course. There is of course 

ah unavoidable conflict built into this approach. On the one hand, 

it was our intention to establiiih what the skills and learning 

capabilities of future TlOl stiiclents'^ will be immediately prior to thexr 

If 

entry into the course ^ On the other hand, we wanted the information 
sufficiently far in advance of the conm^cement of the course for it. 
to be useful in the preparaticm of the instructional material to be 
used in the course. In prc^ctice this meemt that the study had to be 
carried out two years before the course Is due to be presented to 

the first students. Any attempt to base the design of new instructional 

* * 

material on the known skills and abilities of ^future students seems 
likely to encounter this problem unless the ^candidates are carefully 
selected for theJ^ suitability to undertake the course. This latter ^ 
approach is quite jBOBmon, but it is arguable that stud^ts are then 
being matched to the teaching material - an approaich which is quite 

34 
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The problems posed and the teaching material provided in tho 
questionnaires were each presented in a pure, abstract form divorced 
from any practical application. But technol(2j3i^ts use mathematics 
as a tool' for tackling real problems. Consequently, ih TlOl 
mathematics' will be introduced and taught in the context of practical 

♦ 

applications. On the one hand it way make it easier for students to 

learn the .basic numeracy skills if th^ can see their relevance and 

usefulness and apply than to concrete examples. Alternatively, 

studoits may encounter -additionalSdifficulties as a result of having 

to learn how to interpret and remodel problems so th^t they can be 

S 

<^ 

handled mathematically. Consequently ^ the* findings of the Numeracy 
Study can only be taken as a very rough guide to the likely performance 
capabilities Aid learning difficulties of students actually studying 
the course* 

It has been assumed that the student ^pups tackling the various 
questionnaires vere evenly matched in terms of their skills and t 
learning abilities and thdt comparisons between groups are thus valid « 

r 

It is impossible to be certain that the groups are matched unless 
individuals are assigned to th^ on the basis of their known skills 
and abilities. Sii^ these are what we wish to measure and to do so 



' we have* to divide the students into groups first there is clearly a 

conflict here. The best we can do is assume that within each of the 
three educational categories identified, students have approximately 
equal capabilities and skills. They can then be assigned randomly to 
groups in such a way as to «isure that the relative proporitons of 
students from each category are the same in all groups 
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. ' Finally, it has been assumed that the responses obtained are 
honest and accurate. It. is bonceivable that some stud^ts may have 
cheated by looking up the answers and filling these in instead of 
working thijfrfugh the problrais set thetoselves. Some may believe that^ 
despite our" assurance to the contrary, ^ high score would favourably 
influence their future assessment record in the University. 

Each of these four assumptions can be' regarded, as a weakness of 
the. method used in the study in that they cast doubt on the 
reliability of the data obtained and the validity of the conclusions 
drawn there from* However^ before passing judgment, the method has 
to be reviewed in the context in which it was devised. The aim of 
the study was to obtain only a first approximation of the likely 
skills and learning capabilities of potential TlOl students. The 
findings are intended to supplement the Course Team's teaching 
experiences in an area beset by anecdote, hearsay "and prejddice. As 
explained in the Introductiont Numeracy Study is one part of a 
three part strategy for ensuring the coordination of students » needs 
with course production. other two parts, developmental testing 

awl complete course evaluation, will provide the njacessary checks 
on the findings of the Numeracy Study. The strength of the Numeracy 
Study is that it is predictive. It provides information about 
students* likely learning needs sufficiently far in advance ,to be 
useful in the preparation of the course. The other two methods are 
necessarily post hoc^ In their orientation. Th^ can only provide 
information about the suitability of material which has already been 
epared. 

In conclusion, we believe that the findings of the Numeracy' Study 
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demonstrate the Importance of carrying out a i^stematic evaluation 
ot critical learner skills as a pre^requisite ^o the design of 
instructional material. Without the knowledge generated by the 
study, the TlOl Course Team would have been in great danger of 
selecting an initial skill level and subsequmt learning gradient 
v^ch wouid be too gx^eat for at least ^ of the new students for 
whom the course is being prepared. It is hoped that other 
course desicpiars at the Open University and els^Where will be 
encouraged to undertake simil«ur types of Investi gat ions • ' 
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NOTES ON THE EARLY STAGES a^- COURSE PLANNING 

by Phillip© Duchastel/ Roger Harrison, Euan Henderson, 

Barbara Hodgson, Adrian Kirkwood, Robert Zimmer. 

* 

HOW TO USE CONSULTANTS SUCCESSFULLY 

by Judith Riley , . • , . 

DISCUSSING AND EVALUATING DRAFTS ' 
by Judith Riley 

Mi 

ASSESSMENT 

by Judith Riley 

' ■ 

INTRODUCING NEW FACULTY MEMBERS TO COURSE PRODUCTION 
by Phillipe Duchastel and Roger Harrison 

IIHIDGINO THE OXP BETWEEN T^CHING AND LEARtlING AT A DISTANCE 
by Michael Nathenson , , 
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